Video Compression.
Video/audio compression techniques for motion video are becoming more prevalent in computer networks and in video-related services such as those provided by cable companies, telephone companies and satellite or fiber-optic-based service companies. To illustrate, consider the cable television companies' announcements of new cable TV networks that will provide over 500 channels due to the use of video/audio compression. As another example, in the computer industry the current buzzword is multimedia, which combines text, still-frame pictures, audio and motion video. Such systems usually employ some type of video/audio compression to reduce the data storage requirements associated with digital motion video. Standards groups are even attempting to create standards for compression techniques, such as MPEG (Moving Picture Expert Group).
ITVS definition.
To date, it is primarily the computer industry that has been the first to adopt use of multimedia. However, the advent of fiber optics and even new digital data transmission techniques over cable have prompted the cable companies and cable/telephone consortiums to begin publicly discussing new networks that will incorporate multimedia data. To further enhance today's cable TV networks, the industry has also been proposing that the user have the ability to interact with the multimedia data on these new networks. Thus, the term being used for one of these new networks is Interactive Television System ("ITVS"). It is commonly understood that an ITVS will incorporate multimedia data, however, to be precise, such a network might be termed an Interactive Multimedia Television System. In fact perhaps it is even more appropriate to delete the "Television" portion of that term making it simply "Interactive Multimedia System" since these networks will deviate substantially from the conventional medium of television. For consistency, this document will use the term ITVS although it should be clear that this term refers to an interactive multimedia network that uses multimedia data.
Some features that will be available from such an ITVS include, without limitation:
Video-On-Demand, home shopping, electronic data services, games, banking, educational programs and more. Video-On-Demand may be best compared to electronic movie rental, wherein the user may order any program at any time for viewing at their leisure with full VCR-like control. In home shopping on an ITVS the user is not subjected to the broadcaster's schedule of real-time viewing such as current systems. Instead, most likely through a graphically-oriented software interface on the TV screen, the user is guided to the merchandise of interest wherein photographs and motion videos of the item may be viewed and the merchandise may be ordered. Games such as those compatible with the popular Nintendo and Sega models will also be offered electronically and interactively, possibly including interacting with other remotely-located players. Electronic banking, database services and interactive educational programs are also envisioned.
Burst transmission.
As can be seen from the previously-discussed features of such a system, motion video and audio is a major component of the multimedia data. Even in today's compressed formats, the quantity of data for digital compressed video is substantial. This is further complicated by the other types of multimedia data (text, pictures and so forth) as well as the interactivity. Research has shown that all of this precludes a practical system that would deliver such data to the user in real-time. For example, it is costly and inefficient to develop a system that processes and delivers to hundreds or thousands of users just enough data for a digital compressed movie from a library of thousands of stored programs in a Video-On-Demand application, in realtime, as Users watch the program. Instead, the multimedia data should be segmented into portions of the entire program and burst-transmitted to a buffer at or near the user for use at their leisure. The advantages of such a burst-transmission delivery over a real-time system are numerous and will become clear as the description of the present invention progresses.
Difference between a data network and an ITVS.
The design of an ITVS and it's data-handling processes are significantly different from conventional data networks. The present invention addresses the special 5 requirements of a system that incorporates interaction with multimedia data. Of paramount importance is the time element associated with the use of such data. Due to the users' interacting with the system and the time-dependent data such as video and audio, the timing for servicing user-based requests becomes critical to the satisfactory performance of the system. This is in contrast to current data networks. In a conventional data network for example, it is of little consequence if a file transfer requires 2 seconds or 2.5 seconds. However, that 0.5-second difference in an ITVS could very well result in the interruption of playback of a motion video, thus interrupting the playback for 0.5 seconds with a frozen frame on the screen or even worse, random noise. Thus, the present invention describes a system where such time-critical operations are efficiently addressed.
Lang patents.
The Lang patents (U.S. Pat. Nos. 4,963,995, 5,057,932, and 5,164,939) have taught the use of video/audio data compression to deliver video programming in less time than that required to view the program. In other words, Lang has taught that rather than using compression to achieve more real-time channels in a given bandwidth, that compression and bandwidth may be used to burst-transmit the material for viewing and playback control at the user's convenience. Lang does not address the inherent difficulties in an interactive multimedia system, especially one that is designed for a large user base.
Yurt patent.
Yurt (U.S. Pat. No. 5,132,992) has taught a general method for accessing multimedia data from a central library, specifically by using a unique ID imbedded in the stored, compressed, multimedia data of the library. Yurt has taught one method of ordering, accessing and transmitting the requested multimedia data without addressing the unique problems of managing a multi-user network for ongoing interactivity with a multimedia library of data.
The present invention is a system architecture and its related suite of data handling processes.
Thus, prior art teaches a new utilization of video/audio data compression for burst transmission and a method for identifying and retrieving selected programs. The present invention defines a total system architecture and a set of data handling processes which address the unique problems of large-scale interactive multimedia networks. Using a distributed control architecture, predictive software based on stochastic models, segmentation and burst-transmission of said program segments, and dynamic coupling between subsystems, the present invention describes a full-featured, efficiently-functioning ITVS.
Dialog Segmentation and Segmented Bursting
A key aspect of the present invention is a new ITVS data handling method that includes segmented burst transmissions. Bursting of a block of data to a `black box` is said to occur when it takes less time to receive and store the bits than it takes to `consume` (use) them in or via the black box. For example if all the bits representing a 120-minute movie are received by the Client set-top box (the Client represents the end-user component) in a ten-minute burst then we say that bursting has taken place with a Burst Factor (`BF`) of 120/10=12. BF is a measure of such resulting temporal compression and benefits multiplicatively from the transmitted bits first being spatially compressed by any one the many existing algorithms, such as MPEG. Therefore if the movie were spatially compressed at 100:1 prior to transmission, it would take 1/100th of the time for the Client box to receive them thus resulting in a Burst Factor of 1200. The same arguments hold for audio data and in extension, data representing any multimedia program. As another example consider the case of an interactive program, such as a video game. If it takes 30 seconds to download into the Client and the User averages 10 minutes to play the game, then bursting has occurred with BF=(10 minutes.times.60 seconds/minute)/30 seconds=20. This new and generalized concept of bursting can be applied to all types of receivers of transmitted materials for which the time to receive and the time to consume can be defined. Such receivers, of course, must have the ability to store the bursted transmission.
The alternative data handling method, which is well-known and over which the present invention claims distinct advantages, is transmission via bit-streaming wherein the incoming bits to a `black box` are received and immediately consumed at the same speed at which they are received. Such a method has Burst Factor of 1, i.e. there is no burst transmission. Bit-streaming is the mode of operation for the existing class of massively-parallel video servers such as the Oracle Media Server from Oracle Corp. Such devices are being designed into some of the currently-discussed 500-channel cable TV systems. Such bit-streaming systems may be seen as simply digital versions of the analog transmission systems which are currently employed by cable TV distributors.
Bursting an entire linear program (e.g. movie) or interactive application is termed `comprehensive bursting`. In the previously-discussed example, the spatially-compressed 120-minute movie would be comprehensively bursted in a six-second transmission. If such a movie is represented by one GigaByte (1 GB) of data, then the Client system must provide for that amount of storage. Depending upon the design of such a comprehensive-burst system, the entire movie may need to be stored before playback (consumption) may begin. For a number of reasons it can be shown that dividing a large program or interactive application into smaller, logically or semantically compact segments yields economic benefits and improvements in quality of service to the User. Therefore, the present invention teaches the pre-segmenting of program materials and interactive applications, and the transmission of such data in a `segmented burst` mode.
The benefits to ITVS operation from segmented burst data handling are many. Primarily it permits the reduction in redundant storage costs at all store/forward nodes in the ITVS. A Hub connected to many Client systems can store the initial segments of many more movies and applications more economically than having to maintain a comprehensive inventory of all available interactive products. For example, when a User requests a particular movie its initial segment can be quickly downloaded with minimum start-session latency. This is due to the minimized Hub transmit cycle time, which is the time required to cycle through the request for all waiting online Clients. This cycle time is minimized due to the smaller segments that are quickly burst-transmitted to the Clients. After initiating the User session the Hub will then place a timely order or sequence of orders to the Bunker for additional segments based on current Hub memory availability and operating policy. Thus, segmented bursting supports `just in time` ordering, inventorying and transmission to provide similar benefits that `just in time` ordering, inventorying and servicing provides in established commercial operations.
Segmented bursting is seen to maximally unload the upstream nodes from incremental User demands while the `program bits` are being consumed at the Client system. This is another fundamental concept of the present invention namely the ability to maximize the autonomous operating periods at every store/forward level in the system. This is achieved by every store/forward node locally assessing upstream availabilities in concert with the predicted needs of its downstream nodes. For example, in home shopping a User may wish to see sweaters from Sears. The `Knitwear` portion (termed here as a `web fragment`) of the Sears interactive catalog would be segment-bursted to the Client in a few seconds and then consumed by the User in perhaps fifteen minutes. Based on knowledge of User demand and intra-application branching the Hub could prepare for the most-likely User demands by pre-ordering from the Bunker and Sears online services the presentation assets (which are the video, audio and data components of the interactive multimedia program) for pants and shirts, for example.
This illustrates another important aspect of the present invention, namely the ability to order and assemble the dialog segments or web fragments `just in time` at the lowest possible downstream store/forward level in the ITVS. this permits the efficient storage and fast transmission of only the dialog components needed that respond to thousands of individual and concurrent User demands. Such a data-handling method also provides for maximal reusability of all dialog components, especially the costly video and audio presentation assets. These benefits are obtained without having to store all materials at the Hub (or any other) level and then remain online with a dedicated channel responding to every User interaction during an interactive session.
Thus it is seen that the invention will allow the assembly and integration of a very large scale ITVS consisting of several layers of store/forward nodes that can be operated by different commercial enterprises specializing in their own interactive product or service. Furthermore the entire ITVS can be made to operate without requiring the storage of all user-available inventory at any one node at any given time and the further requirement for all knowledge about the state of the system to be contained at any one central location. This ability to effect distributed control with only local knowledge will reduce ITVS's complexity and cost, and increase the ability to provide large inventories of interactive products and services at high quality of service levels to the end User.